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ABSTRACiC 



This is one of" the outcones of the work of the 
Hassacniusetts Evaluation Service Center 'for Occupational Education 
(ESCOE). The purpose of the work described in this report is to 
provide a set of specific behavioral objec^vef^for woodworking 
teachers to use in the development of their high school curricula. 
Special emphasis is given to a discussion on the rationale behind 
criterion-referenced testing. The remainder of the document is 
devoted to a description of the development of t^e objectives and the 
test item bank for woodworking, and gives the following information 
to the teacher: obj active code (block/unit), objective number* 
- objec tive, final product, evaluation criteria, and a description of 
the operation. ""Also included is a detai3.ed llst,,of materials needed 
to test each objective and a 25-item bibliography. The document 
concludes with two appendixes on pretext materials and block and unit 
breakdowns. (BP) 
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I'- I'll. IM^KKORMANCK TEST t)EVI'l.()mKr^l' 1'^^^ ^] ^ 

■ . - • • ... . , \ 

V. - . • ■ ■ ■ ' \ 

Ba«kgrouii.l 

fi.._... , ^ . ..J , . 

I'h' 1968 Ar.itnilin.M.i to the Federal Vocational-tischnical Education Act 

mandated rhe developmunt. of 3tate-wide evaluation systems for the admlnistra- 

■ - ■ .. ■ I 

tion and operation ot feder^ally supported vocational education. Parallel txi 

this mandate the Research Coordinating Unit director for the ConanqnweaUfi of 
Massachusetts was in the process of completing some prideelgfi activi,tie8 for 
the development of a vocational-technical education management ' information 
:iy.;teiu. liy 1969 the predesign of this system had moved Into' the feasibility 
st.jgcs -ind sperif Icat ions of the system were being developed. 

Ac this stage of development New York StaLe, vhlch already had a fine 
c<^ntrali7.od testing program, became interested in the philosophy espoused by.,. - 
'the Massachusetts system and joined in the funding of a roo.e intense feasi- 
bility test, which , 'eventually became the source of the Performance Test De- 

/ . ; 

t ■ i 

/velopment Project.. The Evaluation Service Center for Occupational Education 
(ESCOE) was funded in July 1971 and was I Roused • in Amherst, Massachusetts, to 
ttsL the feasibility of systems develo^ent based upon the principles of (1) 
lot ill ^anrrol and development \f vocational curricula, (2) data-based feedback 
basiu iijV.ri tailored pert orraance tests, and (3) carriculum description through 
i.t;r.Tini;u oehavior objectives. The following report deals with a subrompuncnt 
■>f t!K' hi.'JOE system whi !i vas designed to develop performance tusty as ;; ;ft- 
varu supijiut for the.EbCOE program, 
l^iats ar.d Whys of Performance Testing 

testing is more a new reality as opposed to a new i on( tipt in 
educational tesiing. The concept grows out of the need felt by <• ',u-.aturs to 



sample aAiial jjerf 6rniaiK es t>t trainees ^« opposed to <i|erely measurlryg sjmiptoin 



of 'deXtTed\ <or intendeid) coiiii)et9^ncie& through paper arid pencil tests an< 

• j . \- : \ ' 7. ' ''^ ' ' ' •■ i 

relying upon tt>e precjfictive; powers' (i.e., previously elstablished associl 



thf 

ipot^ tJ>e pred^ictive ; powers' (i.e., previously eistablished associ/atifons 

of paper and pencil test scojres to some hypoth* ti'cai o;r observed criterion of 

' ' '■ "I * ■ ' • • If" 

competeficy in pecforaance) of. the tepi% to infer competency acqui^itiod/ Thi. 

i ' ^ M ' . \ ' ll 

felt need has grown in part ifrotn the iAa^ility of standardized achievement 

^ \ ' . ■ • i'.- , . ' ■ : • ■ / ■ ■ 

tests to deal with the unique objectives of a specific educational program, 

.; • V • ' \ \ I • ^ ' I 

in part from the reportedly low corfrelatfons between measured ski lift and on- 

the-job (or in~t|ie-8hop) t>er,f6rmances, arid in part frpm the lack or realifim 

involved in the, paper and ]|>eh^il testing jsituation. 

* Hence the performante teat can be conceived of as a crlterioh-ref eiipnced 

■ ' ■ • ■ \ ■ 1 

test, in that (1) it is objective or critdlribn-centered (in one-tfo-dne corres- 
pondence with the extant component of a stljit^d objective); (2) iL seeks to 

asceriain a subject's possession of a specific competeficy ra the tf than to com- 

■ 1 . * ' /■ • 

I . r 

pletu a comparison* of the subject's cbmpet^nfcy level to a previously measured ' 
no nil )f(roup; artd (3) it usually requires a ^iQhotombus decision kfs to whether 
the cumpettency -has been demqnstratedi The performance test can be construed 
to be a special^ case of the criterion-refei^enced test In that 
inite atrempt to establish fidelity between the sample obsetva 
formance test and the p erfo r mance being Bdmpledt ~ 



(here is a def- 
ion of the per- 



In the evaluation of iTistruct ional programs in vocationaihtethnical edu- 
cation, the concept of performance testing is especially apprdpriiQte for sev- 

\ 

eral different reasons. , Firsts performance tests cdn be hypothesized to pro- 
duce more relevant and valid date concerning the Instructional program output. 
Vocational program objectives tend to deal wit^ competencies whicM fequire 
concurrent behavior changes across several domains of instructional objectives. 



Ili-Ui r ilii- li 4 ompMshiiii hf ol .1 v/ocatlonal obJi»i-tivi^ m.iy ilopiMid upon llu lovul 



upnu*iiL ol a psychomoLor skill, the ma 



lit^ry of a cognate process, the acquiring 



components, performance tests possess 



ot some fundamental facts, and the development of a parti tutd^ attitude. Un- 
like paper and pencil tests, which eiabhasize -the measurement of the cognitive/ 

aspects of the performance or observations which eaaphasize process and diCti^ 

— / 

the potential to measure the mixture of 
behavior domains appearing in the desired performance. ^ The performance test 
can thert'fore be argued to offer a valid means of measuring Intended outc^pes. 

Second, perfonnance tests produce producif records which can be studied by 
teachers to diagnose the place in the instruction where a weakness may have 
occurred, aiding considerably their ^bility to analyze their instructional \ 
metiiodsr Since the teacher can determine what aspects of the competency arje 
missing, he can trace the point in h|ls instruction where his objectives wer^ 

not met. Also, since the product is' concrete it can be kept longitudinally N 

j / ' i > 

to analyse pupil growth at dlfferertC stages of a multi-year program. 

Third, the nature of the data. j)roduced by performance testing contains 

the t iexi bill ty demanded by the infprmation needs of an evaluation system^ 

The tests are constructed in one-td-one correspondence to stated objectives, 

thus enab ling-select ion of test copiponents from a d r ^ f R h^rk ^1* fitiM -ftrr-i-rr^urh 



a maniier <is to tailor the testing to th^measurement of a unique set ot pro- 

/ . ' '•'"■^ 

grajh obje tivns. Since the tests^ are obJettiviEr^p€^£t4;^^ 

/ 

aspects an instructional program are possible. Since the tests are crltcr- 
ion-rof croi;; ed, skill attainment in a particular area of interest can be as- 
certaL;u a; hence output of instructional programs can be 'i^escribed rclativ'e 
to pt rc a>;e of skill vievelopment . 



Kt'sii.mits o n Te/«t IK'vr lopmcnt, / 

.' - ; ' j ' _ ■/ • • .. / 

IIm- <li slgn/of rhf |)t;rfonnan^(L.|b?st8 had_tii_^^^ both thv. __/ 

•philsophteaL the f»perational sj/ructure of ESCOE. At times both of these | 
structures sf^n/ed as restraining aind occasionally frustrating hurdles for the 



t^st development teasi. 



4 The E/nllosophiciil nature of /eSCO$ provided the foundation of principles 

■ ■ / ^ ' 

which are believed td have caused the performance tests tp be unique. Since 

•• " ■ I / . 

the objef tives were generated /by each local school, several very similar ob- 
jectives appeared for a single behavior within a subject. Dr. David Berliner 
Hjow with j/fie Far^ejt Labo/atory for Educational Research and Development, 
invent^gcT;! proce,ss Jo .sl'ate these similar objectives into a synthesized form 
•Wompanied by item changes providing f or^ the unique characl: eristics of each 
objective. Thus, if enough objectives from different schools were collecttfl 
to represent the curricula, by synthesizing those objectives one could arrive 
at a statement of all desirable behaviors within one curriculum. 

The raw objecti es based upon the curricula of each of the participating 
schools were synthesized to identify the major behaviors within a curriculum 
area. Hence, if the process worked ideally withi, a curriculum area a linear 
y.c't or behaviors was produced. The degree to which this process failed to. 
proii.ce &u':h a linear array ot behaviors comprised thi first major restraint. 
If a singular listing of behaviors could not be gain^^d. then singular test 
items could not be written. 

A riecdhd philosophical principle which developed into a restraining f a« - 
tor was the decision to test only locally-maintained objectives within a 
spot if l<: program. This principle actually Involved several implications for 
testing. First, a student would be tested only on the objectives maintained 



by t <urrlciiluni hi' wiis rtiaeivlng. Therefori", l\w l^st I teniH/ hatl^ttr h,. <ic. 

scyibod In ^ form Indlcattng one-to-one correspondence with the synt host zed 

/ - ' \ - / • 

objectives ^'so that the local teacher could select only those/ items maintained 

/ / •■ ' 

for his course. This selection pattern, however, did Increj^se the logical 



assumption that the tests possessed high validity in regarc/ to th6 courses 
for which they were designed to measure outcomes. SecondJ e< h item had to 
be independent in its ability to be administered, since previous or adjoining 
items would not necessarily be administered with it. Th^s Item^ independence 
served as a restraint to test development in that qbject/ives could not be 
clustered into tasks invoJ.vlng several £e8t items. 

Ipe third restraint involved .both philosophical ahd operational aspecjts 

u ' ■ -I ■ ■ 

in th^t two forms of scoring were preferred by the two cooperating states. 
Philosophically, the state coordinators differed on the location of sco/ing^; 
this disagreement became a restraint to test development in that the items 
developed had to be scorable both in the local school and at a central test 
center. Three forms of scoring meeting this restraint were adopted, with 
choice of scoring form depending upon the nature of the individual item. 
Two of the fbnns are i ised upon meeting the restraint with -a single scoring 
process. The uhlrd form requires two different processes in order to meet 

; 

the dual scoring restraint. 

The .scoring approaches requiring only one process are (1) the caliper or; 
mechanically scored form and (2) the selection of correct response form. ? In 

A 

the mechariito^lly scored approach, several measured settings can be placed in 
a test scoring kit; the student or teacher records by label which of the set- 
tings fits the final product. A kc^y of correct setting labels can then be 

referred to, producing a dichotomous score for the product in term^ of size 

/ 

/ 

/ 



tolerances. In the selection of correct response approach, c( 
can be applied directly to the students' respopses. In both peases either a 
central otfice pr an Individual classroom teacher, can use the keys. 

, The third s|ioring form is not as simple, since two types of scores are 
required to ipeet the dual-use restraint. This scoring form is necessitated 
by the many tasks- in the vocational curriculum which require expert observer 
judgment for the det«?rmination of performance quality. The two types of 
scoVing needed for thesf- itans are (1) structured criteria for observation, 
and (2) pictorial r««cords (color-coded to facilitate central scoring). /The 
structured criteria for obse/rvation communicate to the teacher w^at aspects 
of the product to check in/order to judge the performance successful. These 
criteria would be used irVclasB. In the pictorial s'loring process, camera 
angles ha^3<^ been descr llVed .whych would allow Polaroid pictures td be tal^n 
as records the finis/hed product. Color-coding the criteria checks would 
enable observers In a/central location to determine the quality of the per-" 



formance . - I . ' ' 

•iach of these three approaches provides a meftns through which crediblie 
and unbiased scorei can be obtained. All of the processes can be scored by 
indiviaual tcachci/s and used within a classroom setting without Lhc aiu n 
central scoring Jtatiun. The fourth restraint to test developraont ar;sL-. a» 
this point, sincte it is impossible to arrive at an imme?liately usable set <if 
norms through the current scoring system dnd the dichotomous item response 
without inplem"^tation of a program designed to gather enough data to norm 

the tests. / . 

Two other restraints were present throughout the test development proj- 
ect, both operational in nature. First was the quality and quantity of the 



heluivior.il obVcri iv*vs f hi'mse Ives • Few if any pf the curriculum areas were 
kully descrltpd, and the tests developed are limited to described cu'rriculum. 
In two test areas, more items were developed and the syntheslzatlonr process 
was repeated in order to sharpen the synthesized objectives. In these cases 
much curriculum had been left undescrlbed and the flll-ln process aided con- 
siderably in explaining the descriptions • However, complete and multiple 
ets of items were not available from each school ; therefore the test items 



liiay be lacking In content validity in cases of consultant-written items* may 



be fbpresentative of several behaviors 



or represent only a small segment of the previously unwritten curriculum 



The secotid operational restraint/ 



i and mBf hence be difficult to test 



was that of time. Although the budget 



was small, the Seriously close deadlines lUt development work made time an 
even greater restraint. Creativity is sometimes especially evasive under 
deadlines and within the constraints of administrative conflict • Still, the 
time dimensions were met in terms of design. Since schools were clos^a dur- 
ing the critical month of June, illustrations of some Items of (he tests 
cuuld not be produced; therefore only plans, item descriptions, materials 
descriptions and administration instructions could<4>e developed. 

A final restraint can l^e observed in the language in which the^ proposal 
was writti-n. First, several terms apparently changed in meaning or in rele- 
vannu to the project once development began. One apparent change occarr* i 
in the description of sixteen tests for four areas. One test for ea^; l^vol 
of a curriculum area cannot be developed so as to be equally relevant to all 
schools. Since the schools maintain different objectives, different items 
/ must be assigned to each school, even on the same level. * Hence a more ap- 
propriaLc process becomes the development of an item banH from which tailored 
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^8t rc^n- Fel^ve lo^c^ I or each indls,idxi^il ~?ro9;rii^r iiemdri^ l-rmcrr^ 
atralnts and the dlffenMicea in figure of curriculum required different klndn / 
of tryouts. making the language of the proposal seem some^^imes inappropriat^,. ' 
Purposes of the Test p pvelopment Project ^ ^ ' \ 

The design of the-tast development project included not only the goal of . 
produang tests as products but al^o the goal of establishing feasibility of ^ 
the test development effort acrossj a Inroad spectrum^of vocational-occupaMr^nal ' _ 
curricula. For this reason four different areas of vocational curricula were 
selected for test development. ' Th^a four areas differ^ in hypothetical ' 

difficulty of test development. Thi| arears chosen were.machine shop, wood- ^ 

working and carpentry, electronics, ^nd automobile mechanics. The automobile; 
mechanics area was hypothesized to b< the most difficult since manufacturers 
determi^ied the curriculum, which therefore differed across competing manufac- 

rs. .■ . V,^ : . I / 

The performance tests were hypothesized to be suf f iciei^tly flexible t^o 
fulfill many purposes of a comprehensive evaluation^ system. Because of their 
proximity to the desired outcomes, performance tests were hypothesized. to 
serve as (1) student diagnostic aftd prerequisite instruments. (2) diagroj^jtic 
instruments for the analysis of instruction. (3) criterion instruments. (4) 
measures of classroom achievement, and (5) ptogram success indicators. Each 
of these uses has already been piloted to ^om^ extent. 

The performance tests as developed have several application conveniences. /. 
First, since the test itans are paralleled to synthesized objectives, compi'ter 
selection qf test items or "synob" comparison of items can be used as a meth- 
odology for tailoring tests to instruction. Second, since the conceptual 
frames of the tests can be described, each test /has built-in potential up- 



tuirers. 
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dal in^; oi oxft^nsion l>v I tuv cl.iH^iiroum teacher. 
Problisns Kncountcrud 



/ 



Problcros occjHTed from thr^e vi^polnts^ Flrdt was the problem' of la^ 



of known^Hrectlbn, a handicap which often^occurs In the area of development* 
-Second was th^e problem of lack of perfection ^r completion or the objectives 
used as raw m^aj:erlals fpt^ the development of test items. Third was the prob-^ 
leaf of contend\Lng with dual scoring requirements and with several different 

•/ \ - ■, .. — . ■ - ■• ^' 

kinds of progi;^ emphasis at^d structure. 

■ ; • . . ■ ■ ) ^' 

The f irst problem has been emphasized recently with thfe development work 

■ / ■ ■ 

done on c r iterion-ref eronced tasting. From a conp^ptual point of view, the 

criteria previously used to determine the quality of norm-reference^ tests 

can no longer be used for criterion-referenced tests. < Since the measurement 

strategy ol the criterion-referenced test and the performance test Is to de- / 

termine the possession of either a skill or the papabilit^C^^o carry out a^- ^ 

act ivlt> or process, the degree to which the test differentl«t«i between 

" • ■ \ ' . ' ' 

subjects t:iklngVhe tests does nothing to indicate test qua-lity. Unlike the 

norm-referenced test , in which measurement strategy Is^ to dlstingiiish between 

subjects, the performance test cannot be hypothesized ^ produce large dlf- 

ferencts across subjects nor can any specific level of difficulty be expected. 

Hence, average levels of difficulty and large differences between subjects do 

not indicate quality of the performanci^est • 

In performance testiiig, some concepts of reliability still appear useful, 

while others appear to have lost their relevance. Reliability over time,^ or 

test-retest reliability, is still meaningful as long as the time between tests 

did not include opportunity for the subject to acquire the skill in question 

Since performance tests are designed so that each item does /not necessarily 



refer to thu »ame skill or activity, reliability Indices dealing with h„„gc- 
nelt, ot the te^t no lonRor appear to be relevant criteria for test quality. 

The degr'eo to which the Items of a perf ormance test c,a»e. the^'l^i3*.-of y 
an 4.. and approxlmat.c actual rtquired performances operates in a similar 
relilousblp to the performance test as that of a prediction .ndex to.a norm- 
referenced test. Th_ls..d<;gree of similarity might b. c»pared to the concept 
of lidelity so often used in the recording industry. 

The second problem area Involved the qualityof the raw:materlals used. 
AS should be expected, the syntheois process does not apply evenly to all 
arcs .md was not applied with the same consistency to each set of obiecilves 
in Vne machine shop curricula area, between 70 and 80 percent of the content 
was described by the objec Lives. These objectives possessed adequate de?th 
- a.ros. ...kill areas to enable the synthesis process to produce clear Isyuf .^- 
sizod ab:ieet(ves describing unique performances. The creation of 1 -:™s ■ ■ 
'parallel Lo Jhe synihesUed objectives and possessing the ,indepe»4?^e. and 
tlexlbillty required by.the phllosophyof, the systi was a Straightforward 

pro(^ess. / 

In tne woodworking area, between 60 and 75 percent of the content was 
describe.!. Unfortunately,, the synthesisers of the raw objectives failed to 
Ptcd,>c.: synthesized objectives which dealt only with single perfcrmahc.s. 
Uste..>l. .he raw objectives were synthesized by similar or relAted behavior, 
and .ne product of this proce7, w* a matrix o^ similar perfor-nances (rather 
than a single performance) with several form c|.|ges denoting differences In 
condition, and extents across schools". Sinc^ tRse products .eemed usable, 
the decision was made to produce a matrix of Ltj items gen«at«Lin^ one-to- 
one correspondence to the performances included in each synthe.l....! objectl 



This decision yas the. source of some time lost due to the expanded number of 
test Items which had to be written; however, this increase In Itanswas ac- 
companied by a large increase In test specificity, which increases the degree 
to which the performance- |:est can be tailored to fit a given Instructional 
program without any noticeable loss of efficiency of the item banking process. 

Dut? to the varlanp6 of material dnd the limited scope of the objectives 
developed for. the electronics curriculum area, a decision was made to rewrite 
many oif the synthesized objectives. For more than one-half of the contract 
perlbd two of the test development team members struggled to find a format 
wlthlrN*t**«*i the scope of the electronics curriculum could be /^ttt^ci^ed . By 
expanding the number of /conditions it was found that classes of performance 
could be described by/ synthesized objectives. Hence, through considerable 
redesign ap.d a smali set of compromises of the synthesis process involving 

uniqueness of pe^forniances and allowance of performance form changes, sub- 

■ , . ■ I - .• ■ - - 

collections of electronics objectivcs^co^l,4 be written wjjich would allow 

test development along similar conceptual lines those followed in the 
development of the/ machine shop test. Results of the test development ef- 
tort again produced item banks, as in t^e two previous test areas, with the 
items possessing similar relationships to the synthesized objectives. 

' In the are.i of automobile mechanlcy^, less than 50 percent of the content 
was described by the raw objectives. Many of the subdivisions of content were 
vtoo spa^t? to allow for the development of synthesized objectives. In addi- 
tion, the synthesis process applied seemed irregular across blocks and units. 
'""I he lev/el of abstraction of behaviors described by the raw objectives and the 
int»!rdepen<J('nce uf the performances raise questions concerning the appropri- 
ateness of Lhe synthesis process in this area. Certainly, the limited number 



• / 

/ 
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of usable syutho.i7.ed oijectlves m.! the ne/^sany revisions of the existing 
obiootivcs made the decision to, rewrite jr^e Objectives essential. Revision 
or th. curr,cula.n dosc.iptions were m/e in relationship to the job orlenta-. • 
Lion of the curilculan. Test items,/were written around standard mechfnlcs ^ 
t.sk. as described in the automo^'ile mechanics curriculum. In so.«e of these 
it:.r.us, svuthesized obj ect ives^/e Vtested in a. forn^at which includes a . luster 
of the ubjoctives provided ^ the ESCOE system. In other items, only parts ^ 
of E.SCOE-pr.,iuced objecti/es are included in the new synthe^ed objectives 
hciug ce,ted. Once a t/st item has been constructed, the process can be re- ^ 
,,,r.,.Hl ch .t ..vsten/capability as achieved in the other three test areas ^ 
can h. 6ai...1. • Pyause of their time-consuming nature, tasks in the ourricu- 
l.."sucl. a. disa/sanbly or reass^bly of a motor 'or transmission were not In- 

.ora/le test items. Instead, either sample tasks extracted fro. 
L)>.. la.,;. uaWMgeable task or written or pictorial selection it..us were. 
lo le-.L iiiese bhas«^8 of the curriculum. 
rUo tnxrd probh-mjarea encountered was the difficulty involved in the 



cx.i.^icnce of Lv.o s^pa.rite scoring requirements and in the time limitations 
or 'the' test development! project. It was not always 'possib le to produce useful 
'•n^.iass s oririg of the performance item and credible, objective centralized 
\ :.c.nin.. or ch. perfornahce through application of the same scoring process. 

M. r.Lore .on.e Lterts arl suspected to produce more useful scores in the . lass- 

than in a central $coring situation, while the reverse is suspected .1 
. C.ur .( en>.. Only time land study of the tests can alter or af fim these .uk- 

1 

pi,.oa-.. I. Ls anforturl^te that systematic refinement of the woodworkin,. . 
.iLVtronlcs, and automobile mechanics tests is not planned to occur along 
the ;.;r.e UneR as those applied to the machine shop test. 
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The following report includes development and field testing procedures, 
Item bank descriptions, recommended analysis procedures and uses tor one of 
the four test ar^as briefly d^s^tibed above. 



/ 
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I. HATiONALE FOR DEVELOPING Tilt; ' 
OBJECTIVE AND TEST ITEM BANK 



Ono of the tUitvgs which is badly needed in the area of vocational . 
education is the availability of behavioral objectives with which teachers 
can systematically construct thfeiTcarrieula^ In_ addition, testing mater- 
ials arc needed in some flbcible fo^at such that teachers and/or external 
examiners can assess the extent to which individuals and/or groups are 
achieving the specific course objectives.' The flexible format is neces- 
sary so that the testing materials can be used for individual assessment 
as well as program evaluation on any specified subset of objectives from 
the bank. Individual aissessment takes the fonp of providing informacion 
to the individual and the teacher on the particular objectives the student;^ 
hari mastered i hat appeared on the test. Program evaluation would consist 
ot ry|JortIng, in part, the proportion of examinees in the student popula- 
\i(}n ot interest who have mastered each of the specific course objectives 
I ovtu-eJ on the test. 

Tho availability of an objective and test item bank would laciiitate 
teachers nnd evaluators making criterion-referenced measurements to assess 
students and programs (Hambleton & Novick, 1972). To do this the test 
score inlormaLion should relate to specific program objectives and be in- 
terpreted relative to prespecified performance standards. (In a later 
section of this report we will discuss additionaKf eatures of criterion- 
referenced testing and measurement;. Also we will contrnst norm-referenced 
tests with criterion-referenced tests.) ' ~ 



v. 

/ 



j 2. PURPOSES 



— ' --"'^le purpose of the work described In this report is to provide a 
set of specific behavioral objectives for woodworking teacherb to develop 
their high school curricula, and to provide materials necessary and gulcje- 
linesN^or teachers and evaluators to cons tryct-xTTter ion-referenced perfor- 
mance tests. The objective bank is complete to the extent that it covers 
synthesized objectives previously developed by the ESCOE staff. (However, 
it is wet^l-known to all involved in the ESCOE project that many woodworking 

^jiroas are Aot covered. It is expected that as these objectives become, 
available in the future they will be incorporated into the materials de- 
scribed later in this report. Fortunately, this can easily be done with 
the system we have developed.) 

Basically, what we have attempted tb do is specify what an examinee 
has to do to demonstrate mastery of- specific objectives. For' each task, 
examinee performance is graded as either acceptable or unacceptable by re--, 
ferrirtg to the evaluative criteria. We ha>^^e provided guidelines for doing 
this and reported on ways to do both individual assessment and program 
evai'jatioa. A special feature of our material is that all testing conalsLs 
of students (.ompletl ng performance tasks. Also, the flexibility of the 
tcrstlng materials shoyld make them more useful to teachers in the ^ichgulirr^ 
Lu should be noted that little attention in our performance testing has 
been directed toward process. This clearly has nothing to do with the im- 
portance we attach to it since we consider process extremely important. 
Rather, in this first stage of test development we preferred to restrict 

\ 
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3. \ DESCRIPTION OF MATERIALS 

v)ne of the major Htumblihg blocks in developing woodworking peVfor- 
maniv tasks from. tlje syilthesized objecliives was that in many cases theV . 
synthesized objectlv.^s covered several different skills. Hence it was \ 

■ ' ! 

Impossible to develop only a single performance task to cover each syn- 
thesized objective. While it wo\ild have been possible to write several 
tasks, the mapping of each task |:o the particular skill within the syn- 
thesized objective would not be ^lear* The best solution seemed to be 
to rt'write each synthesized, obje^itive into as many specific skills as - 

i 

necessary., | 

Uther problems included th^t synthesized objectives were not vnrltCen 
as carefully as they should have been, so that much time was given to re- 
writing them. After several %oor" synthesized objectives were removed 

: / ■ - 

and many others rewritten, 1^5 specific objectives were written to t'^place 

thf.t 32 synthesized objectiVes. Writing a performance task to measure each 

/ . 

oi the 165 objectives ther) became a fairly straightforward task* 

Oitr task Included the development of the objective and test item bank 
for woodworking, presented in Section 7, with the following inf Oi.niation: 

objectj.ve code (block/unit), objective number, objective, ti^al product, 

K .^^-^-r- 

evaluation criteria, a^d description of operation. The infopn^tion wasX 
cpllectcd on each objdctiye. * \ . 



The resulting matbrials were read by five woodahop teachers on two 
different occasions and many of their suggestions were used to modify our 
materials. In ddd^lpti<jB representative sample of 54 objectives was pire- 
tested with more than 60 students, and here the, results were encouraging.' 
Details on the pretest are presented in Section 5. 

A. USING THE OBJECTIVE AND TEST ITEM BANK 

The objective and test itan bank can be used for: (1) individual 
assessment, and (2) program evaluation. 
IndLvidvtal Assessment 

It is possible for a teacher to select ^any set of objectives that he 
desires for testing. (The teacher mayselect common objectives for stu- 
dents or different objectives, depending on his goals.) Then the list of 
objectives that an individual can c|o-and/or cannot do can become part of 
his record and the information can be used to effectively monitor the stu 
dent thrrough the program. ^- 
Program h:valu|ttton 

\\ If the list of objectives on which we desire ihformation is small 
eiiough, thedSJ-t may be possible to administer each objective to all stu- 
dc^i^ts and by summing across individual rfeports we can calculate the pro- 
porVion of examinees who perform each objective correctly. 

\ln the more typical case, we desire information on all course objec- 
tives but the time required to test all students would be prohibitive. 
In this situation, evaluators should resort to the technique of an item- 
examlnoe sampling. Such a technique has been used successfully to eval- 
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Tlu |>sychometri( technique of 1 tera-examlneL' s;iiiipling (perhapa '>t!tbur 
known as multlple-mati-ix sampling) for estimating tust score parameters has 
recently become very popular with researchers and ev.iluators. Item-exam- 
Inoe sampling could l>e described s^^d a technique where, according to Lord 
and Novick (1/968): ' 



M no nover lapping random samples of n binary ittang each are drawfi 
(without replacement) from an n-item'test and treated as separate 
subtests; it is not required that m = n/n or that M » N/N.- A 
different Subtest is administered to each of M nonbverlapping 
i-andon samples of N examinees drawn from a population of N 'exam- 
' inees. " ' , 

f 

Lord (1962) was the first to estimate the test parametevs (mean, varl- 
ance and frequency distribution) for a populatiali by utilizing an item- 
exafliincf . sampling design rather than adminiaCeriiiB the total test to a 



Rumple ui fXiunlnees. More recently, dlffqfrent researchers have investi- 

' L ^' ■' « ■■ 

«ated v.iirious Item-examinee sampling designs to detferraine the "optimum" 
ntun'Dor of ittrts, and examinees to use in different testing situations. 

Curri. ulum evaiuatur^ are presently usii^ item-examinee sampling de- 
signs to collect more information on a program than would be available 
JJ .-rl 1 ^ x.imLnees in the population of interest took only a subset of the 
total nur.ber of items \n^the "pool." item-examinee sampling is particu- 
larly applicable for curriculum evaluation, where information about the 
group is usually of more interest than information on individual examineib. 
Lord and Novick (1968, pp^ 252, 257) report on several other situations 
where itan-exami nee sampling would be useful. 

Final result Includes a set of proportions to indicate the proportion 
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of oxamitut-i? who surre.ssfullv completed each '<>b j.-. t • v.' . • ^ 

In the next four pages^we will describe onr mode I, that has been used 
successfully to evaluate instruction and seems appllrable in vocat ionaV educat I 
'^i'*"-B-'"*^'helnsl ve Achie v anent Monitor i ng (CAM) 

r ' ~ r-. - f . 

Hambleton, Corth and O'Reilly C1972) describe a model for the eval- 

I 

u.itlon of student achievement in i lassrooms and for curriculum evaluation 
called C fjnpre he nsivo Achievement MonitoKlng (CA^) . A 11. of the decision- 
making is done on the basis of criterion-referenced test results. The 
CAM design includes the following components: - 

1. The definition of a curriculum with behavioral objectives; 

2. The writing of test items . to measure student performance cn each 
. ' objective which are criteri^o^-rtif erenced test items; 

J}. The organization of a set of randomly parallel tests, where each 
^^est is madie up of items measuring all of a sample of all of the 
objectives in the curriculum and therefore' represents item sam- 
I>Ung; 

4. Tiie design of longitudinal usually every three or four weeks* ' 

- — V •! 

schedule of test dpcasions throughout the cou^"se; 

5. Th^ analysis of the test data and the re|>orting of- results by c 
computer, usually within a couple of days; 

I 6. The interpretation of the' results by evaluators, teachers and 

• ^ ■ 

students as a means for making better decisions about rhel'r in- 

struct ion and curriculum; and 

7. The modif icat,ion of curriculum, instructiunal activities and tlie 

CAM design based upon the results. 

The CAM methodology has been designed to work well with any grade 

level or currlculai? area. In fact, it has already been used successfully 

in more than 20 schools, with more than 15,000 participating students, and 



at gnidp levels from third to twelfth and in evc*ry academic subject aiiM. 
(Hamblc/ton, Gorth and O'Reilly [1971] provide a dt tailed report on one ot 
the many applications.) 

Particularly important to the success of the evaluation is the use of 
the computer. It alleviates the frequent^ encountered bottlenecks of^ 
most evaluations, i.e., the analysis of data and the. reporting of results. 
The coinpute^^llows maximum freedom in the design of evaluation, which CAM 
has used by incorporating longitudinal testing with item sampllngt 

The inf oripation which is provided in thef CAM system includes: (1) 
for individual students, (a) the total score on the current test and all 
previous tests, and (b) information on the correctness of their response 
to each item corresponding to course objectives on the current test; and 
(2) for any subgroup of students and ah/ set of questions after each test 
administration,^ (a) the achievement level on each objective, and (b) 
achievement y^^rofiles which display graphically the ley^l of achievement 
on all objectives on the previous test occasions. 

The computer allows students* achievement to be plotted on any .^iven 
objective (or group of objectives) for the entire course*^. This plot, 
called an achievement profile , g^ves a graphic presentation of/the changes 
in group achievement through6ut the course. Achievement prpfiles are a 
unique type of infonnatidn available from the CAM model. ^ . 

Figure 1 presents hypothetical achievement profil^ for four ob|ec- 

tives from a course. In th;is example, objective 1 wa^ taught between the 

■ ' ' / 

first and second test administrations, objective 3^etween the third aiu! 

fourth testings and objective 4 between the f ourt^^ and fifth. For the 

•/ 
/ 

reason given below, objective 2 was not taught. / On the pretext in the 



example, aU object ivos except .^umber two show achievement at Lhc chaace 
level or about 20% on the f iy^lopt-li^n ami tipie-choice ^ items . Using the . 
achievement profiles after the second test achninlstrat ion the following 
decisions might be made: (a) objective 1 was not loarhe^ and should prob- 
ably bo retaught^ in a somewhat different way; (b) since the performance 
level on objective 2 was high on both tVfe. first and second test adrninis- 
tratii'us one could safely skip instruction on it. After Che sixth test- 
ing, on the basis of the CAM data the following decision could be made: 
the porfonnance level on objective 3 is slipping and since it is an im- 
portant objective it should be reviewed. It is also noted that the per- 
formance level on objective 1 has not^ changed. One might postulate that 
the topic is just too difficult for this particular group of students • 
In simimary, C/\M represents an- application of criterion-referenced 
testing to program evaluation carried out using longitudinal testing and 
the notion of item-eximlnee sampling. Such a st^Vategy might be applied 
to analysis of vocational education data. 

\ 

Test Construction / | 

Using thb procedures described below performance tests can bej constru(/ted 
easily. Sample tests and evaluation sheets are reported in Appendix A. 
test, iten, evaluative criteria and testing materials required corr^spony^lnj^ to 
each objective are foiind in sections 7 and 0, The examiner prepar-^js three 
lists. The first is made of objective statements to compose the test for 
the students. The second list is made up of evaluative crlterl.i coded to 
the objectives in the test. The third list is made up of the mnttrials re- 
quired to conduct the testing. The last list, Is of course, used by the 

- i 

examiner to set up his materials in advance of the ttst administration. 



5. DESCRIPTION OF THE PRETESTING 



In order to determine the f eaaibillty Vand practicality of our mater- ^ 
ialsr^r^atesting of 5A objectives was carried out in three schools in the 
Amherst area involving over 60 students In t^ie ninth and tenth grades, 
during the latter part of May. Copies of th| two forms of the test and 
the teacher evaluation sheets which were useti are reprinted in Appendix 
A. To some extent t€^cher8 assisted in the selection of objectives to %)e\ 
covered on the tests. i 

It was found that the testing went most ismoothly in the school? where 
the teachers wer« well-prepared. That is, iiii order for the test adminis- 
tration to rup smoothly we found, and hardly a surprising result , that it 
was essential for the teacher to prepare materials necessary for the test- 
Ing in advance. Also, it seemed to be tmpdrtant that students be informed 
of the purpose of testing.* Testing went mote smoothly when this had been 

done. ; . 1 

The results of our pretest could be analyzed in several Ways. ,J|or 
example, it would be possible to report the proportion of' students in tfach 
nUdss who had mastered each of the objectives on the test. Comparison of 
proportions on similar objectives across different groups could be dohe to 
identify the more aff«ttive teachers, schools, districts, etc.. Of courpe, 
such checks as the 'similarity of students on other factors like\intelll 
gence, interests, etc., the appropriateness of the objectives foV the dif*-. 
ferent groups and the eqtiivalance of grading standards in the difycrejit 
groups would need to be considered in interpreting tl^e results. 



Th(* lost results (' uld also be used to dlaguuse iiperiflc disabilities 
oi stiuUiiis. s^uikMUs who have not mastered certain objectives could be 
r'issigned to remedial programs to bring their performances up to standard'. 

Final jjaprossions of the pretesting Include some concerns about set- 
ting tolerances, and whether **do-overs** are to be allowed, VThatever the 
decision, it should be standardliied for any particular administration of 
the tests. * 

6. SOME BACKGROUND ON CRITERION-REFERENCED TESTS* 

In rocent years vre have s^n the Implementation of many Innovatlvi^ 
instructional models such as Computer-Assisted Instruction (Atkinson, 
1968), Mastery Learning (Bloom, 1968; Carroll, 1963) / Indivlduarized In- 
struction (Glaser, 1968), and Project PLAN (Flanagan, 1967). Into our 
jschools. Features suQh as the slpecif ication of curricula in terms of 
behavioral objectives, detailed diagnosis of beginning studentb, the 
availability of alternative instructional modef', individual pacing and 
^iequencing of materials, and the careful monitoring of student progresb, 
are comruon to mo6t of the Instructional models mentioned above and to many 
other instructional models to which we refer In this report* Programs 
along the same general lines are now being developed in vocational educa- 
tion. Within these instructional models, tests are used to establish an 
individual's achievement on specified content, i.e., instructional objec- 

^ Much of the material in this section is from an article by Hamble- 
ton, R. K;. Assessing student progress: A criterion-referenced measurement 
approach. In R. DeLone, Controvaraies in education * New Yorks Saunders, 
1973, 



tivos, nnd to provide information fof making a variety of instruct ional 
1i?vl-!ai«J'^i>« Unf oxtunateiy, the most commonly known procedures for con- 
structing, administering, and analyzing teste, and interpreting scores, 
arc not as useful within the context of these new instructional models, 
and in some ca^es are completely inappropriate. This is because tradi- 
tional norm- referenced tests are specifically constructed to estimate ^^ 
each individual's Ability level and feb permit comparative statements 
(e.g., ranking) aciross individuals. As a result we have seen the devel- 
opment of a new kind of testing, criterion-referenced testing. Criterion 
referenced tests are specifically designed to meet the measurement needs 
of the new instructional models. 

The movement toward criterion^tef erenced testing in the schools rep- 
resents an important developiftent advoCfited by many educators. Such a 
inovtunenL seems particularly relevant in vocational education programs. 
However, many problems relating to matters such as test construct J on, 
inteipretation of test s'-ores, and the development of reporting systemb 
have yet to be resolved. 

The purpose of this brief section of the report is to introduce the 
nature of criterion-referenced testing, to make some distiniftions between 
norm- referenced and criterion-referenced tests and measurements, and to 
describe one use of criterion-referenced testing in an individually pre- 
scribed instruction program. Extensions to vocational education are clea 
However, it should be noted that SOTie of the discussion in this section 
pertains specifically to paper and pencil tests whereas all our woodwork- 
ing- ::;.Kcr ia Is are performance tasks. 

\ 



Cr iLcrii Mi-Referenced Teals; Deflnltlgn and Some RelaLed Problemb 

Tlu' teim 'V:rlL or Ion-referenced test" was Iniroduced by Claser (1963)"- 
to in.ikc the distincLlon between tests designed to compare Indlvidualb and 
tests designed to measure Individual achievement relaMve tu some sped- 
f led. domain of tasks. Of the various definitions proposed for criterion- 
referenced tests (e.g., Krlewall, 1969} Ivens, 1^70) we prefer the defini- 
tion proposed by Glaser and Nltko (1971). That Is, 

A criterion-referenced test Is one that Is deliberately constructed 
to yield measurements that are dlred'tly interpretable in terms of 
specified performance standards. 

According to Glaser and Nltko (1971): 

Performance standards are generally specified by defining a class 
or domain of tasks that should be performed by the individual. 
Measurements are taken on representative samples of tasks drawn 
from this domain, and such measurements are referenced directly 
to this domain for each individual measured. V . 

1 ■ , ' • 

Defining well-specified content domains, developing procedures for 
generating appropriate samples of test items, and setting performance 
standards represent significant problems for measurement specialists, but 
they will not b^. discussed furthef: in this report, although jit should be 

noted that partial answers to problems otle and two have befiinl addressed 

: ■ 1 
\ 

earlier in the repott. ^ 

Uufor*' unately, ibecause of their newtiesB and some unique Iproblems to 

be described later, there is a shortage of information' on crilerioa-ref- 

't ■ ■ 

.erenced tests. However, It is still disappointing to note that 8f:ldom do 
even the most recent educational measurement textbooks include more than 
one or two pages on the topic. According to Cronbach (1970) **Th(e testing 
movement has given too much attention to comparative interpretajtions (to 
individual differences) and too little to absolute, criterion-referenced 
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..asaro.e„t.".. The need for such Information Is .aslly .een when one con- 
siders the fact thnt more and more Bchools each year are adopting Instruc- 
tional models chat require criterion-referenced testing. 
, ^^,„„„ Mnrn. P.oferenc.d and Criterion-Referenced Testing 

Nom::referenceiJtesH avrallable aptitude ana 

achievement tests can be classified as norm-referenced because they are 
designed to measure Individual difference..: The meaning which can be at- 
tached to any particular score depends upon a comparison of that score to 
other score., achieved by students iaklng the test. A standard procedure 
is to report the percentage of scores that fall below particular scores 
lu aiff_erent groups, of \xamlnees (I.e., percentile norm, tables), thus 
maklngLt possible to interpret an l.Kllvldual's score relative to.the 
scores of students In several different group., If desired. 

A norm-referenced te« Is constructed' .pwUlcally to V'taUe the ■ 
variability of test'score^, since such a tfest'li ftore likely to pr<«iuce , 
f«er errors In ordering the Individual, on the mea.ured ability. Since 
nor^-reforenced tests are often used for .election purpose.. It follow, 
that minimizing the number of errors In ranking Individual. Is extremely 
important. 

Lt is a Well-known f a^^ ^hat norm-referenced test, are constructed \ 
-'..ing .«r.nn.smH^n/p4cedures_whe^^ "l-^"-^ " 

•■produce tests with desired statistical properties r«her than-to be rcpre^ 
^ sentative of some content domain (Gronlund. 1971). It Is partly becau^.- 
r.t this fact that the test acotes cannot be Interpreted relatV <r'°'»»- , 
„„uent U.n,ain. BoH. ..asy and difficult test Items do not usually appear 
in norm-referenced tests because they contribute very little to test score 
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variancc. Also, items which do not itreaeure the same ability as the major-/ 

■ . . . . * I 

ity of other items in the test 4re usually removed. l:lmpirlcal evidence 

■ ' • / 

support these conclusions is provided by Cox (1965) • His work revealed / 

that the selection of Items from a total Item pool by the usual item aef 

/ 

lection procedures (l.e.» utilizing item difficulty Indices— proportioijid 
of individuals passing items > and item discrimination lndlC68*-mea8urei of 
the "relationship between item success and ![)erf ormance on some criterion. ' 
test usually the total test dcore) resulted in tests which contained pro- 
portions of Items measuring instructional objectives different from those 
in the total item pool. 

Criterion-referenced tests . The emphasis on mastery learning in the 
new instructional models has lad to an interest by measurement specialists 
itf criterion-referenced testing. Criterion-referenced tests can be used 
to serve two purposes. Firsts they can be used to provide very specific 
information on the performance levels of individuals on the Instructional 
objectives. This information can be useq» for example^ to determine wheth- 
er an individual has "mastered** particular objectives (Block» 1971). To 

aid in making individual mastery decisions on particular objectives is one 

if 

^ of the major reasons for developing the objective and test item bank in 
woodworking. 

Second) criterion-referenced tests can be used to evaluate the effec- 

#1 .. ^ . , 

tlveness of instruction. Norm-referenced tjists given at tfhe end of a 

course are usually inappropriate for making I evaluative decisions on the 

i 

effectiveness of instruction because they a^e not designed to tiover the 
instructional objectives* However » criterion-referenced tests are quite 
useful to the curriculum evaluator because of the specificity of the teet 
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results to the ina^trud tional objectives, ^--..^ 

, • / \ 

lypualiy, Lhi.' itms on a criterion-referenced test -vca» be thought of 
as a sample from some well-defined content domain. Knowing a student's 
test score doe 9 not allow us to accurately say which items were* answered 
correctly, but we' can make a pretty good estimate of the proportion of 
ttans in the domain that he could answer (Popham & Husek, 1969). 

What are whe appropriate procedures for constructing criterion-refer- 
enced tests using itiOTs presented in multiple-choice format? It should be 
clear that since a score on -a criterion-referenced test is compared to • 
some performance standard rather than to the performance of other individ- 
uals, for the test to be a good measuring instrument it will be necessary 
to change the itedi selection and test construction procedures. 

Since comparisons among individuals are of little or no interest when 
uslog a criterion-referenced test, it follows that a test constructor is 
not usually concerned with developing a test to maximize the variance of 
test scores. Therefore, a tefitL developer should not use traditional item 
selection procedures described earlier to choose items because they were 
specifically designed to produce a test with maximum test score variance. 
For example, criLerion-referenced tests are often used either before stu- 
dents are taught specific instructional objectives or Immediately after, 
students hdve been taught specific instructional objectives. In the for- 
mer situation, most students will answer few or none of the test items and 
in the latter situation, they will answer most or ail dt the Items, it i^ 
clear then thftt in both situatlonis very little variation in total test 
3core'? within the group of students will result." Consequently, the u^ual 
indices for assessing the discriminating power of an item will be very 

♦ 



ERIC 



-17- 



i-lose to zero, which is considered an Indication ot a'.poor test itun in 
cons true Ling a nurm^reCerenced tent. ThuSi there Ih the dl8tlnct| pobsl- 
■ bility.ihat if traditional item. Selection procedures developed fcir use 
with norm-re^ferenced tests are applied to the probl^i Of con&cructlng 
criterion-referenced teats many good items will be unwisely discarded. 

It would sem that what is needed now is a test theory developed 
spet-lf ically for cr iterion-yef erbhced tests. Some progress has been made 



in this direction by Cronbach and Cleser (1965), Krie.wall-X1A6L9-)^..|^ 
and Nltko (1971), and Hambleton anrd Novick (1972). 

Criterion-referenced testing is somewhi^t dlffejent when applied t« 
the woodworking area since our items consist of per^formance tasksf riather 



re' are 
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than multiple-choice Items. Hence some of the issues described abo 
not relevant to our materials. . . " 

' A summary . Although a test cannot be classified as either a norirv- 

referenced or criterion-referenced test by simply Id^king at it, tfhe two 

• \ 5 ■ ^ \ 

kinds of tests ar^ designed for quite different teaaons, and constructed 

using different pi3cedures. The norm-referenced test ls\ constructed using 
traditional item selection procedures for the purpose of m^ing comparl- 
sons among individuals. In contrast, a crltej^ion-referenced ^^st is de- 
signed to facilitate decision-making relating to individual performance 
and effectiveness of instruction. Procedures for constructing the tests 
are only now being developed. 

D;^t)tlnction Between Testing Instruments and Meaauremenf 

The differences between norm-referenced and criterion-referenced 
tt'sts can be further clarified by properly distinguishing between testing 
instruments and measurement. Certainly criteriomTeferenced measurement 
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Is substantially different from norm-referenced measurement. In the lat- 

\ . • . - • ■■■ ' ' - 

ter, one i^ primarily Interested In using the measurements for makins 

coraparatlve decisions among Individuals, whereas in the'' former the meas- 
urements are used to evaluate student performance relative to specific 
performance levels. 

The major distinction between norm-referenced and crlterion-ref or- 
enced measurement is In terms of the kind of Information available. With 
the availability of a theory for making norm-referenced meaaurenients we 
ha ve proc edures for constructing optj^^ 

norm-referenced tests. Then the pertinent question seems to be whether 
or not the instructional models which require different kinds of measure- 
ment require new kinds of tests or the usual tests with alternate procc- / 
dures for interpreting test ^cores. We subscribe to the belief that new u.- 
tests are needed, constructed in ways which are substantially different 
from the traditional approaches to test construction. We dlntinguish a 
criterion-referenced test from a norm-referenced test on the basis of the 
procedures used to construct themt 

It should be noted that a norm-referenced test can be used for cri- 
terion-referenced measurement, albeit with some difficulty, since the 
bampllng of items is such that in many cases objectives will not be cov- 
ered on the test, or will be covered with only a few items. Also, a cri- 
terion-referenced test constructed by procedures especially designed "o 
makii criterion-referenced measurements can be used to make norm-referenced 
meaS\irements. However, a norm-referenced test is constructed specifically 
to maximize the Variability of teat s ores (whereas a criterion-referenced 
t'ejt is not) since such a test is moSt-^lkely to produce the fewest eirors 
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liL ordering the Individuals on the meaaured. ability; Clearly a norm-*re£- 
erenced test can be used to make criterlon**re£erenced measurement , and a 
crlter ion-retferenced test can be used for norm-referenced measurement, but 
both usages will be less than optimal. > 

Thus It. would seem misleading to talk about tests as either norm-* 
referenced or criterion-referenced since measurements ol)talhed from either 
testing instrumenc can be explained with a norm-ref erefrced interpretation, 
criterion-referenced interpretation, or both. But it is clear that a test 



J* 



-^emis4rguGt«llT;£ >>£..jua.klo8,;:; ^^ likely-! 
for the other. 

Peel slon-Theore tic Approach to Criterion-Referenced Measurement 

Our own conceptual framework for criterion-referenced measurement 
goe& this way. Like Cronbach and Gleser (1965)^, we see testing as a de- 
cislon- theoretic process; that is, teste are given for the purpose of aid- 
ing in making decisions. This is certainly true in vocational educatlbn^' 
where we want to assess mastery or non-^stery of students qh specifj/Cf^ ^ 
Instructional objectives. One of the main differences between norm-ref-* 
erenced and criterion-referenced tests is in terms of the kinds of decl- 
sions th.ey are specifically Resigned to make. Norm-referenced tests are 
particularly useful in situations where one is interested in "fixed-quota*' 
sel^tion (i.e., only a certain number of individuals wi-i|. be selected re 
gardless of their qualifications), or ranking individuals on some ability 
tests. Decisions relating to mastery of instructional materil^ls are best 
done with criterion-referenced tests. With a criterion-referenced test 
we have what Cronbach and Oleser (1965) /U)uld cffll a "quota-free" aelec- 
tioh problem. That is, there is no qi^ta on the number of individuals 



who can exceed the c. ut-of f score on a criterion-referenced test. A cut- 
oft score is set tor a criterion-referenced test to separate examinees in- 
to two mutually exclusive groups. One group is made up of examinees wltl> 
high enough test scores (Vthe cyt-off score) to assume they have mastered 

the material to a desired level of proficiency. The secoi/id group is made 

/ 

;■ • ' / ^ ■ / 

up of examinees who did not achieve the minimum proficieticy standard. 

/ 

Establishing cut-off scores is primarily a value judgmefit. 

' The educational goal ik to have everyone achieve ^he standard's. This 



is attempted by individualizing instruction to tne point of providing 

fj ^ . ■ 

alternate instructional modes (Cronbach, 1967), indiyidual pacing and se-^ 
quenctng, and providing various remedial programs. /i 



a 

of 



The primary measurement problem in the new ol^ective-based curricu 
programs such as tl^, ones being proposed in vocational education is one 
detiermininr if Pi, an estimate ot Pi, is great ei^ than Pq* Here Pi is ,the 
"true" score for an indivlduiil i inisome particular content domain. It 

represents the proportion of items in some content domain that he could 

\ / ■ 

answer successfully. Since we cannot administer all itenis in the^^domain, 
wc sample some small number to obtain an estimate of Pi, represented as 
P^. The value of is the somewhat arbitrary cutting score used to di- 
vide individuals into the two mastety ca^tegories described above. 

Basically then, the tester^s problem is to locate each examinee in 
the correct mastery category. There are two kind$ of errors that occur 
in this classification problem: false positive errors, and false negative 
errors. A false positive error occurs when the tester estimates an exam- 
inee'3 ability to be above the cutting score when in fact it la not. A 
false negative error occurs when the reverse is true. The seriousness of 



making a false positive erjror depends to some extent on the structure of ; 
the Instructional objectlvlea. It would sepin t|j«t this kind of error has 
the most serious efftv t. on pirogram efficiency i^en the instructional ob- 
jcctlves are highly related'^ On the other hand, the serlouaness of making 
a false negative error would 8eem>to depend on the length of time a stu- 
dent would be assigned to a remedial program because of; his low test per- 
formance. It should be noted that many students whQ aJle mlBclassified 
are, of-'course, close to the cutt ling score; hence the i/emedial work is 
not completely inappropriate for them. 

» In practical applications of cric«rlon-ref erenced testing it would 
seem that In order to evaluate the test it would be necessary to know 
something about the consistency of decision-making across alternate forms\ 
of the criterion-referenced test (i.e., test^reliaiiility) . Another con- 
sideration Is the validity of decision-making with these tests. The prob- 
lesa of reliability and validity estimation for criterion-referenced tests 
has not to date been satisfactorily solVed; however, there are several 
papers that deal with the topic. Including Bormuth (1970), Hambleto.. nki^ ' 
Gorth (1971), Hambleton and Novick (1972), and Popham and Husek (.969). 
Criterion-Referenced Testing; An Application 

In the final pat^t of this section we will consider the application 
of criterion-reference^ tests to the area of Individual asse^^sment in an 
Individually prescribed .instruction ptogrm. 

A new Instructional model, typlpfal of many being, implemented in 
schools around the countrj^, is the one being used in the Jamesville-DeWltt 
(JD) High School in DeWitt, New York (O'Reilly & Hamblecbn, 1971; Hamble- 
ton, Gorth & O'Reilly, 1972). 
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Thls ninth grade Individualized Inatractlon program In science Is 
organized Into .-rr^.Honal modules which consist of InstActlonal actlv- 
ItU. designed to teach a single taajor concept which has been expressed In 
terms of behavioral objectives. The instructional activities which make 
.l^i^ module ate. subtlvlded ljsto w^ 

Ity packages (UPs). As 4.ch student proceed, through an Instractlonal 
module there are four type, of ieclalon. which must be made about him. 
To 'provide information for decl^ioo-makln* the following criterion-refer- 
enced test, arc administered: I module pretest, a module, post-test, and 

i " - . < 

several LAP pretests and poatteets. ' 

Briefly let us'conslder ea|:h decision separately. As a student be- 
gins to worw on a module, a modjale pretest is administered. On the basis' 
of his test performance, a presi:rlptlon indicating the particular LAPs^ ^ 
that a student needs Instruction on in the module 1. prepared by th. . 
teacher. Such a procedure will eneure that student, will be working only 
on learning exp^ience. directed toward goal*^ which have not been mastered 
previously. The .Module postteet which 1. *lther tht .«.e te« or an al- 
ternate form of the module pretest can be u.ed for prescribing remedial , 
wort for a student, for grading, end for evaluating the iffectiveneae of 

the instruction in the LAPs. 

Analogous to the module -pretests, the LAP pretests are used to pre- 
scribe a set of objectives within the LAP that the student must demon- 
strate competency in before moving on to the next LAP in his prescription 
LAP posttests are used to dete^lne the extent to which students h^W 
satisfactorily completed the objectives of the LAP. ^ 

Decisions relating .to the diagnosis of learning:aif f Iculties can als 



be made from tYi^ criterion-referenced tests/if the incorrect responseii to 
the items have been carefully constructed/ i,e, , incorrect choices are in 
eluded in ah iteili because they are indicative of particular learning dif- 
ficulties. Apparently this systematic construction of incorrect choices 
to mult Iple-clio ice items for the purpose of diagnosing learning difficulty 
has not be6n carefully explored but offers much potential. In addition to 
being an excellent way of extracting more information from a criterion- 
referenced test, it offers a systematic way of constructing choices to 
multiple-choice items. - - 

One problem that remains to be solved for programs similar in format 
to the JD model is the development of guidelines for establishing cut-off 
points (i.e., how many itq:i8 must an individual pass on a criterion-ref- 
erenced test to demonstrate mastery). 
Summary 

In this, section we have attempted to highlight some of the special 

- ; ♦ • 

characteristics of criterion-referenced tests and compare them with ^orm- 
referen<\ed tests. Also, the application of criterion-referenced testing 
to the protvVerav^f Individual assessment in an individually prescribed in- 

L ' 

struction prog rim was briefly discussed. Some pertinent comftients were^ 
int'luded that made specific reference to performance testing in woodwork- 
ing. \ 

It should be clear from this sc^ction that there are still many prob- 
lems that remain to be resolved in developing an operational criterion- 
refurencLid measurement model for assessing student mastery in the sch wol.^. 
It is encouraging to note that at this time there is probably more re- 
search being conducted in this area than any other in educational measure- 
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ment. ilma It- should not be long before pracc^ical solutions are foun.l 
to ilu* problems. \ 
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8. . MATERIA' NEEDED TO TEST liACH OBJfcCi iVh 



ERJC 



1 1 a t I r i a 1 K Ne t * dk*<i 

Ground planu iron, oilstone and oil, 

/ 

iJround duM'l, oilstone .md oil./ 

/ • . ■ 

/ 

tiround knife, oilstone and olV. 

Ground hand scraper, oflstojne, oil, fllo and burnislier 
Circular saw blade, circular saw machine and oilstone. 

3 pieces of 1" x 6" x jointer, thickness planer\ 

• / ■ • 

1 piece of 1 x A x 2/, 2 pieces of 2 x 2 x 30". Try 
zig zag rule, penci^, specs. 

•■ . . / ■ 

2 pieces of 1" x x 2', folding rule, combination 
pencil, specs. ' 



l\piece<of iVx 2" x 3', sketcirof proje'cr , folr^inK /ruli., 
combination gfquare, pencil, straight edgo. 



/ 



1 piece of/1" x 2 x 3', sketch of project, foJding /niJi', 
Try squai";^, pencil, straight edge,.- 

. /' , . 

1 plec/of i" X 2" X 3', sketch of project, foldl/i?; ulc, 
steel /square, pencil , straight edge. 

/ \ 

1 piece of 1" x 3^" x 3', skejtch project, st/aight Lu>'e, 
"folding rule, laydiit board, (|ambini'tion square ,/penc.i I . 

? •■' / ' / ■ 

1 piece of 1" x 10" X 3', skeftch of projt'ct, Vhyout Ut^^V-i, 
straight edge, try square, folding rule, p^ntAl. X 

• \ / ' / y 

1 piece of 1 x 10" x 3 ' ,\ sketch of project, /layout board, 
straight edge, steel squate, folding rule^ ^^d pencil. 

1 piece of 1" X 10" x 3', sketch ot^oject, strip tc'riplate, 
straight edge, folding rule, steeKsquare knd pencil. 

y . . ■ 

.1 piece of l".^,Ji" X 12", 1/r sheet of ft() sandpaper, i/4 ;.iiL'et 
of /(»2//0 satitfpaper, sandlniTDloGk. 

1 piece of 1" X 6" x^', vibrat'or sandor, sandpaper to fit 
vibroator sander IH-fO. 

1 piece of l.Vit 6" x 2', belt slander abrasive belt for satuiur 

/'I/O. ,.- ' r - ^ , 

'A 

1 pi?ce of 1" X 6" X 2', han^ scraper. 

4 pieces of 1" x 4" x 12" hardwbotJ, powdered ^liie , Hmf ,1 i acr 
to mix glue, glue brush, bar clamps. 

A pieces of 2" x 12" x 12" hardwbod, powdered «lue, rotitnii.er 
to mix glue, glue brush, hand8cf<?w clamps. 



)'Jnr, bhir hnu;|i ; |)ar ampr; . 

■ / i ' 

Stock tor bookf,iKe. whjti. Klue, clami-ing blocks, bar clamps, 
stec'lsquart', fhii' bruslL 

Stock f>5r bookcase-, pow(l.urocl gluu, rontaincjr to mix >^luf 
clamping blocks, bar clamps, glut- bnisli, nteel square. 

r-Stoclrfor -ircl. leader, powdered glue, conialner to mix Hue , 
glue brush, rlainp blocks, pipe clamps. J_ i_ : 

Stock for. aVch leader, ^lue brush, clamp blocks. whU«- glue, 
pipe clamps. . . ' I . 

Sco:k for arch leader, ,K.t glue in glue pot, glue brush, 
clanp blocks, pipe clamps. 

• " . . ♦ 

Stock for drawer case frame, white glue, j-hie brush, clamp 
blocks,. bar clamps, steel square folding rule.. 

Stock for drawer case frame, hot glue in glue pot, gha- brusii. 
clamp blocks., bar clamps, steel square, folding rule. 

Stock for drawer case frame, powdered glue, glue brush, dan.;. 
blockH. bar clamps, stirel squars. fo-lding rule. 

Stock for ...nplete book case with grooves, shelf standards, 
shelf stand.Td screw nails, fasteners, hammer, steel snuare, ^ 
folding ruli , back saw, nail set. 

\ 

\ 

Stock far complete book case with grooves, shelf standards, 
•shelf sta.,.h.rd screw nails, finish nails, hammer, steel square, 
folding rule, hack. saw, nail set. 

Stock for complete book case with grooves .• she It standards, 
shelf standard screW nails, flat head Wood screw:^, screw driv.:r. 
hand drill and drill bet hammer, steel square, folding rule, 
nack saw, nail set. ^ 

S„toclc-fDr complete cabin/.t case with grooves, shelf aandards, 
screw nails, fasteners, hanmer, steel square. r.,ldin^ rule, 
hack saw, nail set. / ^ ' 

.stock: for . omplete cabinet case witti ^ro^vc s, ,,,.] ( s l.u.cJ.. rds , 
serewnajls, fini.-.h n/iils, hammer, .t-el ,',uare, fold. n,- rule, 
.lack saw, [ set . [ .. , - ^• . 

Sto.:k for .:<,. plete cijbim.i rase wifn .-roov s, ...elf st ...uLu-d-. • 
screw uail,^ . rlat heJd wood screws, i,:rc-w driver, hand ,irill ' 
and tvlst drill namm^r. steel square, foldui^ rule, had. 
nail set. . " ♦ ' 



Number, 



Matorials Ni^'Uv^l 



Sti>rk lor rompIi'K' ihesL, fasteners, hairriL^r, sLeeJ ^qua^, 
fohl.i \\\\ xu \< s iM i i ^;e t . 



40. 
41. 

4^. 



Stock tor complete chest* finish naib., hammer, Htv%// ..<}n.iri», 
foldlnj* rnli , nail., stt • . 



Stock for^ coini^l rte r 



lesi » flat head ^'oou ^^cvvy^^^\^ml^\\^\\ drill 



1 apiece oi i -x 6, x 2 



.and twist drill, Hcr(iW d r 1 haJm^ur_^_^' l oe 1 s q^irii^-^.u^J.4l44»^>- 

J_.'iiu44^. — ' • ^• 



Willi rtMic^ave enls, wo\ a f i K.-., >riadpaper. 



1 pieces r' x 4" x Z|' nailed toyethtT wiiii/finL wi n.iii . if 
r o nn I 1 1 iiMiC r , n|ii 1 1 ^ e I - 

■ ' \ \ 

J pii?l:e of r* X |b" >. 2' with concavt- ruf'/., u^^od ) i . . aiv; 
sandpaper^ I 



2 piecivs r' X 1' with one.<^nd niiterid tu 

lui ter vise. 



-4^ . 



45. 



'4/. 



48, 



49, 



•0. 



Drop leaf uible u^p wiLh 2 drop loais, drop Itat hin/s 
combination s<|uarL\, foldinj; rule, rn/isel, pencil;. si r\*v driver, 
specs, and tiammeruV 

Drop leaf table toj^ with 2 drop lyats, drop Iraf hin;;i's, t rv 
square, lU^ding rulfe, chlsoL, fip^cs, pciu i 1 , -ic rew (|r i vc r , f 



and hait^iir. 



J>ro^ leaf t 3i::K» lop with 2 dn/p leal:-., dr^Ap leaf hir. t , 
s.teeLl. sauar(t» folding* rule, chisel, specs , penc i i , . ri v. tri ver 
and hammer, v / 



1 door, butt; jiunaeij , htn;;e screws, combination square, ora\.-lnf^, 
for hinge location, chiisel , butt gauge, screw driver ana 

hammer. . i , ■ 

\ I 

\ > . ■ ■ 

I door, butt iilnges, hfnge screws, try square, drawir,; U^y 
hin^e local ioii, chisel!, screw driver, butt gauge and ,.i;nr.;er. 

/ i 

I door, butt hinges, hinge screws, stei-l square, drawinr ff^r 
hinge location, chisel, scfewdr^ver, butt gau^e, and ii/irrjr.er. 

' Specs for a proj/ct, 1 piece of 1 x 6 >; 2', loJcinv; rilo, 
combination sqiiAre, aijid peilcil. 

Specs* for a Tfojoct, i pieqe of 1*' 6*' v J ' i 1 ii.,. r.m^y ' 

try oquaru u^d pencil!. i , ^ ' 

Specs for a project, |l i)ieLe of ]" :, f)" x 2' loldinj.-, nil.', 
steel squart , and pcnjcll. 



ERIC 



^ji. specs fi)r .1 project, 1 piocv ol I" x fi" x 2' >;to»'l r.ipc, 



36. 



■)7. 



38. 



39, 



65, 



68 
69 



Spi'ts for .» project, I piece ol 1" x 6" x 2' stt-el L.ii.l', 
try squan-, hiid pencil. 



Specs for a pro jecl^ .i.,uIuce-aiL--U' ^^-^"2J-5n-m-r-tapTV 

T steel square, and pencil. * 



Spets for a-projeot, 1 pieoe o'f 1" x 6" x 2', zlR-zar, rule, 
combln.itlon square and pencil. 

Specs for a project, 1 piece of 1'' x 6" x 2' zig-zag rule, 
try square and pencil. , 

Specs for a project, 1 piece of 1" x/6" x 2' zip,-za« ruU-, 
stepl square and pencil. 

1 piece of 1" x 6" x 2' wl>tj one edge straigtif and .squari.', . 
try sqviare and pencil. 



i. 



t>0. .1 piece of 1" X 6" x 2^ Vv. one edge straight and sLarc, 

try square, pencil and .specs. 



ol. 1 piece on" X 6" x ;2' with one edge stiaight and <*'quare, 

combination square, arid pencil. 

62. 1 pieq^i of 1" X 6". x 2' with om edr,e straight and squar.], 

combination square /pencil and specs. • 

1 piece i»f I" X 6" x 2' wi^i one edge straight and sruare, 
steel square and pencil. 

. . s ' y 

i piece of 1" x 6*' x y with one ed^e .straight and . ;.iaiv. 
steel square, pencil/and specs. 



1 piece of 1" x 4/ x T, and try square.. 



bb. Any vertical surf ace and^ level . 

6 7* ^ Any flat surface and l^\Jel . 



I milled piece of 1" x x T and try ^qlMar^ . 

I Diilled piece of, 1" x 8" x i' and try square. 

1 -iilUid piecL' of 1" x/»" ■< 1" ,in<. -ul-.m sruar. . 

. ' railed piece of 1" x i,'" x 2' ai,'! sti-ii square. 

7^- Any (.roject that is as; . mi. 1 c.l , Kiu.T,rinL, steel Vquar. , .a 

foldin>; rule. 



/ 



r58- 



Nuiiibcr 
/i. 
74. 
75. 



! I 

• I 



I' 
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I ilrawci IraiiiLf, l>Un|irint» ruii' ,Hui sii.-t;l i<|M.iu'.. 

I iisscnil) I III ciu'st, bliuprtiii , r.uU' and stri'l K(ni.iri'. • 
1 assembli'il bdok casts bjuijp^t^^^ 



76. 
77. 

78. • 

79. 

80. 

81. 

82.' 

; 81. 

84. 

83'. . : 

86. 

87. 

88; 

89; 
90. 

9] . i", 

i 

92. 
93. 
94. 



r 



squares . 

1 completed table, blutrprlnt i folding rule and stee l square. 

I assembU;.il drawer easu frame, blueprint, folding ruli», and try 
square. . ' 

1 assembled diest, blueprint, folding rule and tjy square. 

1 assembled table, blueprint, folding T^e\ and try square. 

1 assembled frame, bluep^fht, folding rule ah^ combination 
square^ \ ^ 

* ;i assijembled chest, blueprint, folding rulo and combination 
square. a 

^ • ■ ' ; : [ , . r- ' 

i assembled bookcase, blueprint, folding rule and combination 
squar?* ■ . ' 

V .j 

. 1 assembled table, blueprint, folding ruK and combinatitMi . 
square* 

I piece of 1" x 6" x 2*, crosscut saw, sqLare and pt*ncL!. 

i piece of I" X 6" 2*, specs, crosscut saw, square and pencil 

I piece of 1" x x 2*, specsy plarking gauge, and ripsaw. 

1 piece of 1" x. 2" x 12", speo^^ and miter box. 

^ I piece of I** x 2** x 2V, s^cs, and mitei^ box. 

V ^1^ piece of r* X 6'* x y\ try square and hand plane 

1 piece of i** x x/l\ specs^ hand plane and level square. 

1 piece of r* x 6"^ I*, specs, hand plane and level Square. 

1 piece of I*' x 3*' x l\ specs , combii/at ion square , pencil 
and hand plane. - • 

■ ' \ * L ' 

1 piece £f X 6*' x 1\ specs, combinatijon square, level 
square , j)enri 1 and hand plane. 

1 piece of 1" X 4" x 1\ specs, hand plane and level s.^uarc. 

y - 



• \ 



^1 ; 



J— 
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\ ... 



98. 
' 99. 
iOO. 
. 101. 

. 10 i. 

lOA. 

/. ^ 
.106. , 

107." 
108. 



1 



Up. 



Ml. 



13. 



BEST COPY mWMll 



Mai cr ial.. . 



r 



plant*. 



«. I*, sptuv;, M'V s<|uan , pencil . l. iaAl 



K pleci* rVx ,A". X r , spies, I'olillnv. nih\ itiu i I \ liauil 
dr i 11 , aiul iwlstj' ilrl II. 



3nd auger bit. 



1 piece uf V X 4" X T, specs , . folding ' rule , pcni i 1 , h.^nvi 

electric drill and twist drill, ^ 

^ ... 

1 piece of I'' x 4" x 1', specs, hand electric drill jjul couiUt-r 
sink. , ^ 

1 piece of 1" x 4" x 1\ specs, folding rule, pencil, drill 
press and twist drill. 



1 piece of 1" x 4" x 1', ^.pecs, drill" ,)ri«ss and tv.i.;u ^ r 



I inece of 1" x 4" x J' , s|.ers., folding nil e , punc i I , ;n.rin,, 
machine and counterborc - ' 

^ K asaemblc'd tabic frami' .iiid Lablt- toj/ , ( ..rr,i'f hlofk-., i, 
drill, twist drills, gltu-, hnndsrrow c 1 -(nipf; . scrt'w.;, r.w 
ilrivt-r and i old lag rule. 

J 

1 piece of \" j 6" x folding rule ini.I hand plant-. 

1 piece of 1" x 6" x 12", foldi;*^ rule and hand scraiu-r 

1 pivci^ of ]" x.f)" X 12", foxing ruK- and cabinet scrapi't.. , 
i •* , ^ / . ■ ■' ■■ 

1. iplero of 1" X h" X 12", /specs, folding: rule", wise aad -.n.ootii 
pljane. 

1 piece of 1" x 4" x 12", specs, vise ami /mooth plane 

1 piece uf X A" X 2', specs, foldfii)' ruic, vise a:al jacK 
plane. 



1 Riefcc of r' X 6" X lh'\ sper 
plane. 

/' 

i piece of 1" x 6*; x 18'\ spec 
trv square* and jack plane. 

1 piece of i" X 4" X 2' , sp^fcs 
and jack plane. 



1 piece of 1" X 6*' :< IS", specs foi Jn^; rule, vise 
jointer plane. 



s, folding nil*:, vise ari<^ ia< k 
i , fo M i rul r , ponr i I , vi se , 
, loidia,:^ rule, pencil, vise^ 



-bO- 

\ , 

Nambgr Mite rials Nc-odfii 
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130. 



lU. I piece of. 1" X 4" x specn, loliUnf, nilc, trv s.iti.M.-, 

vise and jciliiter plao*^ * 

115. .1 piece of 1" x 4" x 18", specs, folding ruhs pemll. 
vise and jointer plane. 

116. 1 piece of J" x 6" x 2', spec^i, try square, folding rale, 
vise and fore plane. 

7 

\ 117. 1 piece of 1" x 4" x 3', specs, folding rule, try squares 

vise and fore plane, 

1 piece of 1" x 4" x 3\ specjJ, folding rule, vise and foro^w 
^ plane. | / 

1 piece of X 2" x 2 \ drawing of moldinR, portabU' router, 
router molding cutters, wrenches and \x\\. 

1 plastic laminated top, portable routtr, lam inatv t r i n i i r 
^ cutter, and wreijchea. 

■ : ■ - V- • 

121. 1 plastic laminated backsplash, portable routqr, laminate 

trimmer cutter, ahd wrenchpa-. 

Vll. 1 completed frame, portable routor, sirai^'Jit cutter, wron liesj 

router gig with fenr,e and folding rulo, 

123. 1 piece of 1" x 6" x 2', portable router, router i utter, 

wrenches, folding rule and router jig wltli fenoe. 

12^* 1 piece of 1" x 4" x 2 V j portable router, cove citter, 

wrenches, and jdg with Vence. J 

12^» 1 piece of 1" X Y' >: 2', specs, jouite.r n.ac'.li i lu* , C-cli:..pi, 

stop blocks, purhXsLick and folding rule 

/ • * . \ 

126. 1 p\ecc- of 1" X 6" x 2', jointur niacliin'?, and push sti'^. 

127.. 1 pieee of 1" X 6" x 2' and jointer. 

128. 2 pieces- of 1" x 6" x 2' and jointer. 



129. 1 piece of 1" x.8" x 1' and jointer. ; 

■ - '- ■ . \ ■ 

4 pieces of vineur stocK, joinfi-r .'^lul (Mill ' . tcw . i ar 



131. 1 piece of 1" X 4" •. 7,' and a joinift^r. , . , ' 

I • 

132. ' 1 piece of i" x 4" ;■. 2'' .itKi .i jointfV. ' 
1.33. 1 piece of 1" x 6" x 1' and a j oi nter. 

^34. 2 sides of a tnjokcase, loMing rule , r I rt ul ir saw, and dado head. 



1 ; 
J/ 



Ma ter j .i i s NimwIlmi 

2 pieces of stoi-k tor ;i r;ib int-L , Id lilliu» t ii le , rinular ,.w 
and iJiUlo In*. id. ^ 

Stock lor a, chest, radial arm saw, <lado liead, and hjldinK 
rule. 

Stock tor sides. of a luilrli, overlic-ad routt r, straight cuf't-r 
anfl^ folding ule. 

^ ... •' 

Thickness planer, straight edge, cleaning fluid, rags and wrencli 

2 pieces of T .x 6* X' 2 ' , stock list, thickne.ss planner 
and folding rule. , j 

■ / 

J pieces of 1/2" x 3" x 3', stock list, thickness, hand screw 
clamps and folding rule. - 

1 piece of 1" x ^" x 3*, pattern template, pencil, bandsaw, 
and drawing', of scallop. ' 

-2-^ i orp . s of _J[/_4,_^.AQ!'., x : Ll'U-CLQmpass^ and dr.winM ot p ro j c t . 

2 piece of 3/4" X 8" x A', pattern and drawing of project, 
banUjiaw and pencil. 

A pieces of 2" x 3" x 30", pattern and drawing of i)ro.i -ci, 
■-bandf+aw and pencil. ' 

2 Pi«:'c;e'' of I" X 2" X 2', drawing, handsaw, and pencil. 

2 pieces of 2" x 2" x 2', drawing of project, and handsaw. 

2 pieces of 1" x i" x 2* ,-, wood func^; (J-.->^anips, drawing ..f,. 
project and handsaw. * 

Band saw and band saw blad^, „. - V - 

A board witu convoxed cuts^ and a spindl* sander. 

A board with scalloped cuts' and a spiniil.' sand'-r. 

A board wit>i arc cuts and a spindle f:ander. 

1 piece of 2" x 2" x 2', mortising machine, f>c lamps, ';Lop 
blocks, wrenches and folding rule. 

: ricct- of 2" X 2" X 2*, mortising machin..', C- , lamps, 'stop 
J locks, wrenches and folding rule. 

I piece of J" )l 2" X 2'/, mortising machine, C-clamps, stop 
blocks, wrenches and folding rules. 
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I piece of 2" x 2" x 2*, mortising 
blocks, wrenches ^ind folding rule. 



Needi»cl 



nachine, C-cJamps , stop 



1 piece of I" x 2" x 2*, tennoner and coper, coplnp hcudti, 
wrenches, stop blocks, try square, and folding ^rvilc. 

1 piece of L" x 2" x 2\ tennoner and coj^er , copin>5 iieads, 
wrenches, stop blocks, level square and folding rule. 

4 pieces of 1" x 2" x tennoner and coper, copin« head, 
wrenches, stop blod^ks, try squ^e and folding rule. 

4 pieces of l" x 1" >i^l2", tennonver and coper, cop in;; heail, 
wrenches, jitop blocks, try square and fbldlng rale. 

Grinding machine, stone dresser, grinding attachment an • pii^nB 
iron blade. 

Grinding macliine, stone dresser, grinding attachment and\^ 
chisel. ' \ • 

^ ■ Mi 

1 piece or 1" x 4" x 12", drill press, machine bit, try K 
square, folding rulcj pencil, and gig for drilling holes, j 

1 piece ut l" x 4" x , drill press, machine bit, rrv s<|uare, 
loiding rule, pencil a^id set up for drilling a series of 
h6U.s. \ 

1 piece ol 2" X 2" x 2', wood lathe, lathe cutting t ; ^ , 
sample turning, folding rule, pencil and outside calipers. 

1 piece of 2" x 2" x 3', wood lafche, lathe cutting lovilh, 
baluster drawing, folding rule, p€;ncil and outside caliners- 

1 piece of 2" x 2" x 3' wood lathe, latlie cutting toels, 
;)iLtern of i^aluster, folding rule, pencil, and outside ralipcl 

^ pieces of 1.1/4" x 1^ 1/4*' x 30" , ^wood J qthe , lathe \ :t:r.,; 
togls, samplr turning,, folding rule, pencil, nnd otit»^id» 
calipers. , - 

1 piece of J'* x 2" x 2\ wood lathe, lafhe rutting rnnls, 
drawing af s^iindle folding rule, pencil and out side ralipefr- , 

1 piece of 2" x 2" x 2\ wood lathe lathe cutting t(vil<;, 
"pattern of >|»indle, folding rule, pencil. ;ewl 'Mit^ ide ^ i p-^rs 
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APPENDIX A 



PRETEST MATERXAIS . 

Twu Sc^nple Tests and Evaluation Sheets 
Constructed from the Objective and Test Item Bank 




1. Sharpen a plane Iron with a ground plane, Iron, oilstone 
and oil. 

2. Sharpen a chisel with a ground chisel, oilstone and oil. 
fSdV*-*— °* stock^lth stock for the project, sandpaper 



.4. Remove the saw marled on concave cuts with milled boards 
with concave cuts, /a file and sandpaper. 

5. Measure ahd markka piece of stock with a bench rule, pencil, 
try square, sttfck aiid specifications. 

6. Measure and mark a piece of stock with a zig-zag rule^p^cil. 
ti7 square, stock and speplflcatibnsrv 

7. Square one end of a board to r6tno.ye bad end with a try square. 

8. Cro^s cut two ends and remove checks wit^ stock, cross cu/ 
eaw, square, and dimensions. \ 

9. Chafer on the end to specifications with stock, specifications, 
combination square and hand plane. 



10. 
11. 
12. 
13. 

4 

14. 
15. 



Square off one end to; specifications given stock specifications, 
try square, and hand plane. 

Select correct size bit and bore' holes to specif icatkons 
given hand drill, drills, 'lumber and speclficatlonfj 

Select correct size drill and drill 4 t^ole to specifications 
givdn an electric drill, drills, lumber and. specifications. 

Select correct size countersink and countersink a hole to 
specif icatiope given a drill press, co«^teralnk, lumber and ^ 
specif icatilina,^ 

Plane a piece of stock to specified thickness wit^ a piece of 
stock, smooth plane, vise and specifications. 

Joint a piece of stock to apecifications given stock. Jack 
plane, square, vise, and specifications. 



'■.I 1.,,,.. .,1.., ^ ,.iuinK Mt stork given jointer nnd pi,.,; 

; , ' ' ' .. i 4. , , . • 

J8. R.,bbrL rdKi- ..1 .sr.uk friv.n jointer an^ 'stock. . 

Jl9. Plane stock to wldth.fcirvea thickness, planer, stork llsfX 
ntoek and hand screw clnmpi ^ ^ ^ ' 

/ 20. Lay out and cut diagonals for turning Riven band'^aw, 
I stock pattern and drawing. ^ 

^'^^'^^ make adjustm^ts' given handsaw 
and blade of desired aUe, ' / * 



22. ^Remove bpds^w marks from^ a board with con vexed cuts. 

23. Remov^ handsaw marks fro^»% board with scallope^d cuts. 

I ' j ' ' 

24. Dress^grlnding wheel,and grind blade given a dull plane 
iron blade attachiijent , stone dresser and grinding whe^l. 

25. Dress grindlni? wheel and grii^d chisel given a dull chisel 
blade atHachm^nt.,. stone dresser Snd grinding wheel. 

, ' ■ ( • 

26. petdrmlne if a surface is fl^t, Using a spirit level «lven 
a surface and level. * - i 

' •■■ . ' - , ,-\ 

27. Petermlne if a surtace Is flat, using a steel square, 
^ven partis of a Job and steel square. 



-68- 



WOODWOUKINC Hl^lJl'OKMANCr. Il-S 



I. 



t'Vtl liiTiHoa 



- 1. 

2. 
3. 

4. 

5. 
6. 
7. 
8. 

9. 
10. 
1 

12. 

7 

/ lA. 

/ 

/ 

/ 15. 
10. 



Cjuiiin}. C'lj'L^ is. r,!/.. v .sliarp am! l.i'vcC Is fl.,L (not routulcd). 

Cuttii^v ojj^c cu^^ . sharp and bevel it. flat (not rounded;. 

.S.niditii, •; : wi'.''. !i„. No sandpaper m.irky ^should "bf sec 

thf thJcKness -,1)0111^ be uniform. 

All saw .md sandpapW marks should be removed , J c.^^^l^ n unij: 
concavt' cut. \^ y^' 

The mt'.js.ired lines on\ the stock should beaccurni,' 

• - • ■ \ .' 
'llicM m<MSurcd lines oii Uhe stock ^uld be ac evu.ii. . 

The ••.vjuurcd line should be ^fccurate. 

!■■ ■ " ■ ■■■ ■ ' ■ ' 

All choi Ks shLHi|d h.jvi-^boon\cut off squ.ire ' i i... ■ 

boa^J .sluiuid bciacViTr^nLe to within + l/lV,''. 

» i. i j' ' . ■ ^ . 

Tti< miier should be planed exaiHly^ 

i ■ . : ■ ■ i ■ - ' . - 

Kiul^^ii.. jl^bx.', :>quar«', smooth and to size. 

Tlie/'holes should be ftu.- cot^rect size, in the c:.rr*Mf ] 
WUh -he 5'-Arrc..t uni'.le to d clean surface. 



I'ho h-'ii's should be the correct ii?e, in tin c 
with t", ■ corrert anplc to .i c'leatl ' surface. 



i^rill ,!•:•;;.«; "should h.ive been set kip correctly \;ith .i 
and siiu coUr|t ersunk hole resulting. 

The stu;k Biif)u.ld be planed flat and stral}.'ht to tl- 
Tl.c^str---^mrl,: w pl^aned straight and sq.,.,,. ,■, 

^1 .li'. ld be accurate and tli.. ♦d,-,,., 

iiiriii .1 rniuj' [rant tyaight and^square. 



n 1 



17. -Jhi 



- N .VL' bc'un scL up correciJy Lo . ( im 
K^i ' r.M' (jc'c.n s^c't up correctly i^. , C t 



BEST COPV WIPM ml 



20. 

"21. 

■'22. 
23. 
2'.. 

2S. 



"^'t I IV "stork sti,,i:i,| • 1 .ir;.....| coi 1 i',- 1 J y ;,iid 'he of » !», f ,»> i ' 

:ilvHk i.llDuUl he ,ii , m.ll.'. 

Cuts .'.iuHihl .uiin.iu' nuii of ilu correct di'pLli / 

TIk- ?;uidt-, should lu ..iju^aud- pf(on'rly with (Ik- correct icc-ibn. 

^Tiu' fiitu and turvt's t>i.*;+il'd l,c smooth apd to nizc or to line. 

nuv{uts...,iiui (M.j-av slioul.l he smootli and to siz.- or fo lino. 

CrindiuV. will'..! .sh.M!|,l |„ (I rcv.s-dr Lit ,uul .s.|n,n-.' I r, sid-s t, 1 v;hool 
with the i;rcu!ui pLtiu" iron h,ivin>; tiu- (orK-ct aiic.li-. 

tirindi:-.,. wh.T] si.uitl,! he dresuuxl. 1 i .,1 ,,nd square to sidi-^ of v/hcel 
and the ground Ihisol. hljade should tiavc-the correct angle.' 




Stud-iu ihcai^d eorrti.tly 



determiiu "i £ material Is flat orinot 



M.J.cr.L .nould eorr^. uy determine if liurlaire/^ .'.quart wit,, e.ii-e 
or not . ! ' • 

I / ■ ■ \ ■ ■ 
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K I Sharpen a scraper with r\ Rroun<t^ hand -scraper, oil <ir>P ', 
' oil file, and burnisher. \ 

2. Sand a pioco of stock with stock for the project , »^<u\lp,'iper 
and a vibraior sandec 

. . ' - \ ■ 

3. Sand a- pitie of stock with stock ior the project, ..^adpapi-r 
belts and a belt sander. 

4. Set nail r; given nailed milled boards and tools. 

5. Measure and mark ai piece of stork with a bench iuIp . 

6. ^ Square both ends of a board to cut to length with a i rv 

square, board with straight edge and a pencil, 

7. Square both ends of a board to cut to length with a combination 
square . 



8. Rip f?dgt' of .i piece of stork pjven stock, maiKirn'. k- ' ' 
hand ripsaw, and dimensions. , i 

J*. CKaiuTer 'wa^t^pecif leal i.ous given stock, specif ical ioii' , levti/ 
' square and hand plane. / 

10. reelect ..'rroct size drill and drill hole*; to ^Vfr.* 
^iven hand drill, drills, lumber and specif ic^txf.rr 



/ 



' r 



11. SiBlcct correct size bit and bore holes to specif{c<t 
v^Tth a bit brace, bits, '^a^^w^er-'^aa^ 

12. Select correct size drill and drill holes to specification 
With a drill press, drills, plumber and specifications. 

13. Select correc/t size drill and counterbore holes to spc ci 1 1 . >t u : 
with a boring machine, counter bore, lumber and specif i caLi Mi^i . 

* / ■ '■ 

14. Joint a piece of stock specifications given ^tock, 

plane, square, vice and specifications. 

lb. Tamper n pter.<» of st'^ck tr specifications giver, stod*. 
plane, ,vlce and specifications. 



/i6. laper sides of stock to specifications given Jointer, stork, - 
C-clamps, stop and push stick and specifications. , 

17. Joint best edge of a piece of stock giv/jointer and a ,iiece 
of stock. 

.J8. Joint the ends of ^tock given Jointer and a piece of stock. 

19. Plane stock to thickness with thickness planer, stock list 
and stock, 

20. La^ out anekcut scallops given a bandeaw, stocks, pattern 
and drawing .y 

21. Set up and resaw stock given handsaw, stock, jigs and drawing. 

22. Remove handsaw marks from aboard with arc cuts. 

23. Set up and drill holes in stocked per dimensions |dven 
drill Rress, bitr stock and gig for drilling seriesSf 
holes . ^ I 

24. Rough turn stock tcv diameter and turn balusters as per sample, 
given stcfck^, wood lathe, lathe tools and sample turning. 

25. Dress grl^tiding wheet'^md^grlSd^chlsel given a dull chisel 
blade attachment^- stone dresser and grinding wheel. 

ftO. Determine if an edge is square using a try square given part 
of a job a^nd try square. 

27. Determine If an edge is square using a steel square, given ' 
part of a job and a steel square. 




Evaluatton SImuh 



t 



♦ \ 



\ 



!• ...^The seraiJer cuttii^i, etlf'.o shaul(l I^hW a iinifofm hooked ed^'/> IinJ her"—" 
sharp. Trial ' scta,pi» should Iscmv'o n\ markg. 
>- ■ .. _ ' . ^" ■■ \ ■/ ' . < - ^ . 

2. Sandiuj^ should.be done with t)h' >^raln yith no sanclpiiper nla^i^^. 
showinj',. Also the thicknessrol iho slta^k should be uniform, 

/ 3. Sandini> should be done with thr Krain wit\i TTcT-rrmripaper mark^ 

, ,^ showinfi. Also^ rhe thickness of tihc stock ^should be uniform* 

• . \ 

4. Student should use correct nailset and the d»cpth of the sx3t naii 

/' . should be consistent with tht; thickness of tf\e stock. \ 

5. Th(J^«ii>is\ired lines should be iiccurate. 
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6. Th^ s^iU»^re.d line shv)uld be ^Lcurnte /and tlie length should lio ..v curate. 



7. The stjYiayed line should be accurnLc and the length sho ild bo .iv,( urate . 

.. . ^ ' • ' " K . ' . ■ - ■ ' 

8. Tlie od}v^ of the stock should be ripped parallel an! tin- '.viui oi lui' 

bonr l rw.>uld t>e wit[»i:i + 1/16". ^ 

i 

9. ilu; cl«a..iLer should \\c st raij^lU , to /.i/i- with an ai iuraLt v • Aiso 

cut ^.i^o,lul be ::wno Uh. ^ ' ; " ' ' 

■ - . . / . • 

•» / . ■ ■ 

10. Itie hwh ;; should bo of the corrct I r,ize, ^ochti'on^ .md anjii^' a 
clt^aii surlnce. 

11. The holes i^hould he at" the corrf' .t nzo Joration, anj^le to a 
clean surface. 

12. the holes should be of the corr*.^ i sii^c, location, :iiul an/;! ' .t ^> 
.. clean surface . 

13. The borici^; maCiii!n-> should bu set t;p ('orrc^. t i> , . : . 
size of counterboi'cd hole.-r— 

.lA. 'il^e stock fihould be planed straight and b^iuarc n tl ^ rit ! Widtii. 

15. Ihc taper sthui.lc! be correct. ' 

16. Jointer should be correctly set aip and th" ' j.v^- 
\ f i ca L i ^ ..'rs . , • 



17. . ITic ( at should b^u of the correM cjLpih with t \iu *'f !>toLi 

strrii^vn and 5'.*ju.Trc to facoi 

' 1 

I 



X 



BEST COPY r^'r'vABii 



IK. ,. 



1.9. 

21. 
22. 
23. 

24. 

.23. 

26. 
27. 



4.>,ll.llf. lite , . ..!,..iVl «',' .1.11 i<> I I , ,', I . 

Ill' |ii ' > (.•iliii 1- •>( I Ik*^ (i|>c r.it i Oil mIhmiIU I)*' ' *)i I 
Uk' ruiii'( l •M/i' block. 



Patlinv lini?s showing With srnuik^)th handsaw cu':;» 

Jig Setup should hv corr6Qt. Si^e of stock :/uou.Ki he* corn rt 
The curves apd c^uts should he smooth. 

Tlic blzc, location, and depth of hole shcmtrt^trp — rrcuract>. - I 
.cuts should be clean ."^^ 

The .size and^shape of baluster should be mntrheci vUh san^nii- 

The ^\rinding wheel should be dressed flat and ^<\\r\\ c to 
sheol and the angle of the ground plane iron sii>i;id l)e corr. 

• ^ ■ • 

The sc^uarcd line shc/uld be accurate. 



Tlie squared line should be accurate. 



i 1 



\ 



.1 ■ 



erJc 



-74.,- 



\ 



y APP^iiNDIX h - 
liLOCK ANT) UNIT BREAKDOWn 



ERIC 



Code 



-75- 

EVALUATION SEf^VICE CENTER FOR OCCUPATinNAl IjJCATION 
'."'OCi^ u,.(\ Unit BredK 
CabiriftJmaKinq and Millv«rK i/. jm./ 
Block Code IJrri-t- 



BEST COPY AVAILABLE 



Bencfr¥6rk' 



02 
n3 
04 

Ofi 
07 

09 
10 



Maintenance 
Layout 
SandihO 
r.lueinq 



"(jfrainq 

Finishinq 

Cuttinq 

Fittinq 

Lamination 



02 



riand-Toul s 



03 



Power Hand Tools 



04 



nachlnerv 



01 


Measurinq Tools 




Lavout Tools. 


03 


Testinq Tools 


04 


Ra^fepr & FrarTiinc '^o . ft 




Sawlna Tools 




ridmo^ 


07 




OA 


SoHha Tool^ 






10 

I 1 # 


^ffloothlno Tool^ 


n 


Lathe Tools 


12 


Planes 


0> 


" \ ' ■ 


0'' 


. ^3Ciprocating S>w 


03 




Oc 


i^Tane 




5o liter 


06 ' 


Floor Sanders 


07 


Belt Sanders . 


Ofi 


. Mail Inc. '-.atnlnf-- 1 


09 


Oi 5C Sa ideVs 


If) . i • 


Finishinq Sdnoers 

. 1 


01. ! 


1 

Jointer 


02 


Circular Saw 


03 


Tfiickfiess Planr" 


rA 


Radial Arm Sav.-i 


05 


Scroll Sav/ 


06 


Band Saw 


07 


Welt Sanders 


Oo 


SpindVe Sar.der 


09 


Disc Sender 


10 


Shaper 



04 



05 



tVAL'iATION SERVICE CtMTER FOK OCCWATICiAu EOUCATICN COP^ ^MMlABlt 



T6W '"filocIT 
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